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r^ lems in terms of superharmonic functions is given. In a general Markovian 

framework, the value function of the impulse control problem is shown to 
be the minimal function in a convex set of superharmonic functions. This 
characterization also leads to optimal impulse control strategies and can be 
seen as the corresponding characterization to the description of the value 
. function for optimal stopping problems as a smallest superharmonic majo- 

rs rant of the reward function. The results are illustrated with examples from 

—I different fields, including multiple stopping and optimal switching problems. 
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1. Introduction 



S^ Stochastic control techniques play a major role in many fields of applied probability. In 

H particular, the developments in mathematical finance have stimulated the activities in 

this branch of control theory in the last decades. Many of these approaches have the 
disadvantage that they lead to non-realizable optimal strategies since these strategies 
consist of interventions at each time instant in a continuous time model. The right 
mathematical framework to consider discrete interventions in a continuous time model 
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is given by impulse control problems. 

Impulse control problems have been studied for for decades. It seems to be impossible 
to give an overview over all fields of application and all different variants that have been 
used. We only want to mention finance, e.g. cash management and portfolio optimiza- 
tion, as surveyed in |Kor99j . optimal forest management, see |Wil98j . |Alv04bj and the 
references therein, and control of an exchange rate by the Central Bank, see |M098j , 
|CZ00j . Most of these articles are based on the seminal work developed in [ BL84] . which 
still turns out to be the main reference for theoretical results in this field. For under- 
lying diffusion process under some further assumptions, the value function is proved to 
be a solution of a corresponding quasi-variational inequality, that also characterizes the 
optimal strategy. A more recent overview over results for jump-diffusions is given in 
|0SO7j , see also |Kor99] for a survey with focus on financial applications. 
On the other hand, it it known that there is a strong connection between impulse con- 
trol problems and problems of optimal stopping. Under certain conditions, the value 
function of the impulse control problem can be found as the limit of a sequence of value 
functions for associated optimal stopping problems, see |0SO7i Chapter 7]. Moreover, 
the value function of the impulse control problem can be characterized as a solution to 
an implicit problem of optimal stopping, where implicit means that the reward function 
in the optimal stopping problem contains this value function itself, see |Kor99] . 
For Markovian problems of optimal stopping, the most flexible and valuable approach 
- both from a theoretical and practical point of view - seems to be the superharmonic 
characterization of the value function; more precisely, under minimal condition, the value 
function is the smallest superharmonic function majorizing the reward function. This 
characterization goes back to Dynkin ( |Dyn63| ) and turned out to be the right formu- 
lation for most such problems. For an explicit solution, this approach can be translated 
into free-boundary problems, which can be solved in many problems of interest. An ex- 
cellent overview over recent developments in this field is given in the monograph |PS06j . 
One of main advantages of considering superharmonic functions (instead of, e.g. using 
a formulation using quasi-variational inequalities) is that regularity conditions can often 
be stated in a more natural way from a stochastic point of view. 

One of the consequences of this superharmonic characterization is that optimal stopping 
problems for an underlying one-dimensional diffusion process can be solved explicitly in 
many situations of interest, since the superharmonic functions turn out to be transformed 
concave functions, see |DK03j . |BLOO] . or |CI11] for recent treatments. Therefore, one 
can say that optimal stopping of one-dimensional diffusion processes is well-understood. 
Inspired by these result, in the last years, different authors considered special classes 
of impulse control problems for an underlying one-dimensional diffusion processes, and 
obtained a solution in terms of superharmonic (resp. excessive) functions, see |Alv04aj . 
|AL08j . and Ega08 . One of the main advantages of these approaches is that they work 



in a very general setting without strong regularity assumptions on the problem, that are 
often needed for applying alternative approaches. 

The question arises whether there is also a general characterization of the value func- 
tion of an impulse control problem as the smallest function in a set of superharmonic 



functions, as for optimal stopping problems. The aim of this article is to consider the im- 
pulse control problem from a purely superharmonic point of view to use the well-known 
advantages for optimal stopping problems also for impulse control problems. This is car- 
ried out in a very general Markovian setting in the following section. The main results 



are Theorems 2.3 and 2.6, that give a characterization of the value function as well as the 
existence and description of an optimal strategy under very general conditions: Under 
natural assumptions, (for the problem without integral term) the value function of an 
impulse control can be characterized as the smallest r-superharmonic function h with 
Mh < h, where M denotes the maximum operator. This can be seen as a consequent 
dual approach to impulse control: The maximization over impulse control strategies is 
transformed into a minimization problem over superharmonic functions. It turns out 
that it is possible to follow a similar line of argument as for optimal stopping problems, 
although some refinements of the arguments are needed of course. To see the connection, 
we use a similar presentation as given in |PS06t Chapter 1]. To the best of the author's 
knowledge, the presented approach is new in the literature, although many connected 
results are already known. Some of these connections are discussed in Section |3j Fur- 
thermore, the theoretical results are illustrated with examples there. For the article to 
have a moderate length and good readability, we illustrate our results on some examples 
only, and give some hints for further applications. The general framework for the results 
obtained in Section |2] allows us to directly identify many other classes of problems as 
subclasses of the framework discussed before. More precisely, we apply the general im- 
pulse control theory to optimal stopping problems, multiple optimal stopping problems, 
and optimal switching problems and obtain the corresponding results for these classes. 
Furthermore, we treat on example with a discontinuous cost structure explicitly and give 
some hints for the solution of impulse control problems for general Levy processes. 

2. General theory 

On a probability space (fi, J-',P) with a filtration {J-'t)t>o we consider a stochastic pro- 
cess X = {Xt)t>o with values in {E,B), where ii^ is a locally compact separable metric 
space and B denotes the Borel cr-algebra. We assume that X has cadlag paths and is 
left-continuous over stopping times. Furthermore, we assume X to be a strong Markov 
process with respect to the family (f'x)xeE of probability measures with a measurable 
time shift operator 6. Without loss of generality, we can assume that the process X is 
given on the canonical space and the time-shift acts as 9t[{u}s)s>o] = {^t+s)s>o- 

Our set of strategies are impulse control strategies; these are sequences 5* = {Tn,jn)neN- 
For general Markov processes, the definition of the controlled process with respect to S 
is not immediate. We only give an intuitive explanation here and remind the reader of 
the formal definition in the Appendix |Xj 

Under the controlled measures (P^)j:^e^ between each two random times r^-i < r„, 
the process runs uncontrolled with the same dynamics as the original process. At each 
random time t^ an impulse is exercised and the process is restarted at the new state 



7n- Here, r^ is a stopping time for the process X" with only n — 1 controls and 7^ is 
measurable with respect to the corresponding pre-Tn a-algebra. 

For jump processes X, the following technical problem has to be taken into account: 
By construction, the process X has a jump due to the control that take place in time 
Tn- But furthermore, the uncontrolled process X" on [r„-i,T„] may also have a jump in 
time Tn if X"" does not have continuous sample paths. Therefore, X^ 7^ -^r„- in general. 
For our further considerations, it will be important to consider the process X" also at 
time point Tn- Therefore, we write 

for the value of the process at r„ if no control is exercised. Obviously, for continuous 

underlying processes, we have Xt-„,- = X^-^^, which motivates this notation. 

We furthermore assume that for each x E E the set A{x) C E is the set of possible states 

that the process X may be shifted to from state x, that is 7n is ^(XT-„^_)-valued. At 

time point Tn, the controlled process is shifted to a point 7n and between two r„ and 

Tn+i the process behaves as the uncontrolled Markov process. Note that Xt-„ = Xt-„ _ is 

allowed. We call this action a degenerated shift and assume (without loss of generality) 

that this is always allowed. This corresponds to the case that no control takes place. 

As usual, we call an impulse control strategy S = (Tn,7n)nGN admissible, ii Tn /^ 00 for 

n /^ 00. 

Moreover, let K : E x E ^ M. he a, measurable function, the cost functional, fulfilling 

Ef f E e-^-^-K-{Xr^,^,Xrj] < 00 (1) 

\n=0 / 

for all admissible impulse control strategies S = {Tn,'yn)n, where K~ denotes the negative 

part of K. We often think of the case that the cost functional K is non negative. In that 

case, the assumption obviously holds true. We want to remark that in our discussion the 

cost function does not only depend on the difference x — y (assuming that E is a, vector 

space) as in many other treatments of impulse control problems, but is an arbitrary 

measurable function of two variables. 

In the following we interpret K{x,y) as the cost of shifting the process from state x 

to state y. Therefore, it is reasonable to assume that K{x,x) = for all x. With this 

convention, it it no restriction to assume that all stopping times Tn are finite a.s., and 

to assume tq = 0,XrQ = Xq. 

We furthermore fix a measurable function / : i? — ;> M such that 

Ef r e~'''\f{Xs)\ds < 00, xeE,S. 
^ u 

Since degenerated shifts are allowed, it particularly holds that 

t-OQ 

eJ e-^^|/(X,)|rfs<oo, xeE. 
Therefore, the r-resolvent / of / for the uncontrolled process is well defined, that is 

fix)=Ej e~''f\Xs)ds, xeE. 



We consider the impulse control problem given by the following value function 

v{x)= sup Ef [ re-"/(X,)rfs-f;e-'"""K(X,„._,X,Jl, x e E, 

where the supremum is taken over all admissible impulse control strategies. 
First, note that for each S = {Tn,'yn)n the expectation 



Ef 



/ /-oo oo \ 

/ e— /(X,,)rf. - Y: e-'-^"i^(X.„,-,X.J 

V-^" n=0 / 



is well-defined in [—00,00) since the first summand is integrable and the second is inte- 
grable in [—00,00) by assumption ([I]). Since degenerated shifts are allowed, we obtain 

vix)>Ej e-''fiXs)ds = fix)>-oo, (2) 

^ u 

so that V > —00. We furthermore assume that v is finite. 

We first rewrite the reward of this control problem to deal with the integral term. 

Lemma 2.1. (i) For all finite stopping times r and all x & E it holds that 

E, re-''f{Xs)ds = -E, {e-'-^JiXr)) +7{x). 

(a) For each admissible impulse control strategy S = {Tn,'jn)n cii^d each x E E it holds 
that 

Ef f r e-^^fiXs)ds - £ e— "K(X.„,_,X.j) 

V^ n=0 / 

=Ef (f: e--" (7(X.J -7(X.„,-) -i^(X.„,-,X.j)) +7{x). 



(Hi) For all X E E 

00 
v{x)-Jix)= _sup Ef^e-'^^"F(X,„,_,X,J, 

where 

Z(x,„,_,x,j=7(x,j-7(x,„,_)-i^(x,„,_,x,j 

Proof, (i) immediately holds by the strong Markov property. 

For (ii) note that the process X runs uncontrolled between each two stopping times 

Tn-i,Tn- Therefore, (i) yields 









We obtain 



Ef 



\n=l-^'^"-i n=0 / 

(CX) CX) \ 

n=l n=l / 

Now note that for each k 

n=l n=l 

By the same argument as in (i) and dominated convergence it holds that 



Therefore, 



Ef 



V-^O n=0 / 

Ef f E e""" (7(x.j -7(X.„,„) -i^(X,„,_,X,j)) +7(x) 



\n=l / 

Since the summand /(x) is independent of 5, taking the supremum over all S gives 
(iii). D 

Note that by (iii) of the previous Lemma, we could assume - without loss of generality 
- that / = 0. But since an integral term arises in many problems of interest, we keep a 
general / in the following. 

Now, we introduce a set of superharmonic functions, that will be the main ingredient 
for our further considerations: 

U:={h:E-^W^h is r-superharmonic, /i > 0, h + J>M{h + J)}, 

where the maximum operator M is given by 

Mw{x) = sup {w{y) — K{x,y)), 

yeA{x)\{x} 

with the convention sup0 = —00. 

Next, we see that each function in Ti is an upper bound for the value function. Fur- 
thermore, we get lower bounds for some special functions. 



Proposition 2.2. Leth:E-^R. 

(i) If h El-L, it holds that 

v<h + J. 

(a) If X E E and S = {Tn,'yn)n is an impulse control strategy such that 

Ef (e— "/.(X,„,_)) = Ef (e— "-i/i(X,„_ J) for all n G N, (3) 

and fulfilling the growth condition 

Ef e-'^"'"/i(X^„ _) ^0 forn^oo, 

then 

ih + 7)ix)<vix). 

Proof. Let S = {Tn,'yn)n be an arbitrary admissible impulse control strategy and x & E 
such that 



E^ 



/ e-^^f{Xs)ds-Y: e^'-^-K{Xr^,^,XrJ > -oo. 

V"^'^ 71=0 / 



Since h ^Ti, by the optional sampling theorem for nonnegative supermartingales we 
obtain (keeping in mind that X runs uncontrolled between Tn-i and Tn under E ) 



Using this inequality and Lemma |2.1| we get 



Ef 



/ e-^^f{Xs)ds-Y: e-'^-ir(X.„,_,X.J 

=Ef (E e-'^-" {7{XrJ-7{Xr^,-)-K{Xr^,^,Xrj)j +7{x) 
<E'Jf:e-^^-({h + 7){XrJ-{h + 7){Xr^,-)-K{Xr,,,^,Xrj)]+{h + 7){x). 



\n=l J 

Since h + f > M{h + /) we obtain that 

ih + 7)iXrJ-ih + 7)iXr,,,-)-KiXr,^,^,XrJ<0, 

therefore 



K 



/ e— /(X,)rf.- E e— "ir(X.„,_,X.J < (/i + /)(x). 

V"^" n=0 / 



Because S" was arbitrary, we see that v{x) < {h + f){x), that is (i). 

On the other hand, under the stated conditions we obtain (ii) by following the previous 

proof. n 



The previous proposition can be seen as a verification theorem. Indeed, it is a gener- 
ahzation (with less-exphcit assumptions) of |0SO7l Theorem 6.2]. Now, we examine the 
structure of the solution more detailed: 

Theorem 2.3. Assume that v is measurable. 

Then v — f is the pointwise m,inimizer of T-i, i.e. v — f El-i and v — f < h for all h ETi. 



Proof. By Proposition 2.2 (i) it suffices to show that v — f GTi. Note that v — f >0 

by ([2]). On the other hand, since immediate control is possible, {v — f) + f = v > Mv = 

Miiv-J)+7). 

It remains to prove that v — f is r-superharmonic. Let cr be a finite stopping time. Since 

V is measurable, so is 



e-^^{v-f){X,) = sup e— Ei Ee-^'"^(^r„,-,X.J, 

S={Tn,'Yn)n n=l 



where the equality holds by Lemma 2.1 On the other hand, using the strong Markov 



property, almost surely we have for each admissible impulse control S = {rnj^jr, 



n=l 



=E: 



Sa 



\n=l 



-TTn. 



-ir(x,„,_,x 



'TriA 



T, 



where 5*0- = {jn,a-,ln,a) is the time-shifted impulse control given by r^^o- = cr + rn0^cr,7r; 
^n°Qa- This shows that e~'''^(f — /)(Xo-) is the essential supremum of the set 



E:; 



5:e— ir(X.„__,X. 



"^n.CT > 



\n=\ 



Ta\'- S impulse control > . 



Following the line of arguments in |PS06t p. 47], it is easily seen that this set is directed 
upwards. By the standard properties of the essential supremum, there exists a sequence 
(5'fc)fceN such that 



E?'" 



\n=l 



By the monotone convergence theorem we obtain 



r.S'fc ,, 



E.e— (.;-/)(X.) = hm E^/- ( E ^-'^^'^^""^^^(^r, _^,X.,„J < (t;-/)(x). 



n 



Corollary 2.4. Assume that there exists an optimal impulse control strategy S = (r„,7n)n- 
Then v — f is the pointwise minimizer ofH. 



Proof. By Lemma [2. 1| we see that 



oo 



{v - f){x) = Ef 5: e— "K(X.„,_,X.J 

n=l 



Therefore, v is measurable and the claim holds by the Theorem 2.3 D 



In the following, we construct an optimal admissible impulse control strategy S = 
{Tn,1n)n, i-G. S = {Tn,'jn)n is an admissible impulse control strategy and for all x & E it 
holds that 

V-^'^ 71=0 / 

To this end, we will often assume the following weak form of the triangle inequality for 
the cost function: For all a; G £^ and y G A{x), there exists some e > such that for all 

zeA{x)nA{y) 

K{x,y) + K{y,z)>K{x,z) + e. (4) 

Note that this is a natural assumption for optimal impulse control problems, where often 
two types of costs are assumed: Fixed costs and proportional costs. The proportional 
costs naturally fulfill the standard triangle inequality. Now, since the fixed costs have 
to be added, the extra summand e is natural. For example, in the survey article |Kor99] 
the cost structure in M was assumed to have the form K{x,y) = \x — y\ + K for some 
i^ > 0, where the assumption Q is obviously fulfilled. Furthermore, we often assume 
that for all a; G -E 

A{y) C A{x) for all y G A{x). (5) 

In other words, (IS]) means that if it is possible to shift the process from state x to state 
y and from state y to state z, then it is also possible to shift the process from state x 
to state z directly. The natural conditions ^ and ([s]) guarantee that if it is rational to 
trade from x to y, then no immediate trading in y is rational: 

Proposition 2.5. Assume ^ and ^. Let v : E —)■ M. be a measurable and x,y G E such 
that 

Mv{x) = v{y) — K{x,y). 

Then, 

v{y)>Mv{y). 

Proof. Choose e > as in (|4]) and let z G A{y). By ([s]) we have z G A{x). We obtain 

v{z) — K{y,z) < v{z) — {K{x,z) — K{x,y) +e) 
<Mv{x) + K{x,y)-e 
= v{y) - K{x, y) + K{x,y) - e 
= v{y)-e, 

hence v{y) > v{z) — K{y,z) + e. Taking supremum over all z yields 

v{y) > Mv{y) + e > Mv{y). 

n 



Now, we come to the second main result of this section, that is a theorem that guar- 
antees the existence of an optimal impulse control strategy. Furthermore, the optimal 
strategy is described in terms of the pointwise minimum of T-i under natural assump- 
tions. The advantage of this theorem compared to the previous results is that it is 
stated in term of the minimizer of H and not in terms of the (unknown) value function 
V. Therefore, no (direct) regularity assumptions on v are needed, that are often hard to 
establish. 

Theorem 2.6. Assume that ^ and and ([s]) hold true. 

Assume that h is a pointwise minimizer in Ti, that v := h + f is lower semicontinuous 

(Isc), and Mv is upper semicontinuous (use), that fulfills the integrability condition 

E^supe-''^\M{v -J){Xt)\ < oo for all xeE. (6) 

i>0 

Furthermore, assume that the stopping time 

r^:=inf{t>0:XtG5}, 5 = {x G E ■.vi.x) = Mv{x)}, 

is finite P^-a.s. for all x E E, and that there exists a measurable function (j): S ^ E such 
that for each x E S 

Mv{x) =I7((/)(a;))-ir(x,0(x)), 

and let the impulse control strategy S given by 

''0 = 0, 70 = a; Pa; — a.s. for all x E E, 
Tn = mi{t > Tn-l : v{Xt) = Mv{Xt)}, 
In = (p{Xr,„-) 

be admissible and E^e^^'^"-h{Xr„) — > 0. 

Then it holds that 

V = v 

and S is an optimal admissible impulse control. 

Proof, (a) For A G (0, 1) write 

Sx = {xeE: Xv{x) < Mv{x)} and Cx = S'i. 

Since v is Isc and Mv is use, we see that Sx is a closed set and 

Sx\S:={xeE: v{x) = Mv{x)}, A / 1. 

Write Tx := inf {t > : Xf E Sx}- Since r^ < oo P-^-a.s. for all x G £" and since S C Sx 
we obtain that tx is a.s. finite under all measures Fx,x G E. 



10 



(b) Fix A < 1. We write 

g{x) ■.= E,e-'^-^{v -7)iXr,), X e E. 

Now we show v — f &%, where v{x) := \v{x) + {1 — X){g + f){x) . First we show that 
for all a: G -E it holds that 

Mv{x) < v{x). 



By noting that M is a convex operator (see also Subsection 3.1), we have 

Mv{x) < XMv{x) + (1 - A)Af (^ + 7)(x) 

= XMv{x) + {l-X) sup [Eye~'^^{v-J){Xr,)+J{y)-K{x,y)] 
yeA{x) 

<XMv{x) + {l-X) sup [{v-J){y)+J{y)-K{x,y)] 
yeA{x) 

= XMv{x) + (1 - X)Mv{x) = Mv{x). 

If a; G 5*;^, by the previous inequality and the definition of v it holds that 

Mv{x) < Mv{x) <v{x) = Xv{x) + {I - X)v{x) = Xv{x) + {I - X){g + J){x) = v{x), 

where we used that g{x) = v{x) — f{x) for x G Sx. On the other hand, for x G Ca we 
have Mv{x) < Xv{x). Hence, 

Mv{x) < Mv{x) < Xv{x) < v{x). 

Since v — f is r-superharmonic, by the general theory of superharmonic functions 
we know that so is g : x \-^ KxC"'^'^^ (y — f){Xr^). Therefore, so is -u — /. Hence, we 
have proved that v — f ETi. 
By the minimality property of h = v — f we obtain 

{v-J){x)<{v-J){x) = X{v-7){x) + {l-X)g{x) 

= X{v-J){x) + {l-X)E,e-'-^{v-J){Xr,), 

i.e. {v — f){x) <E,xe^'^"^^(y — f){Xr^). Keeping in mind that U—/ is r-superharmonic, 
we obtain 

h{x) = {v-J){x) =Exe-'-^^(v-J)iXr,) =Exe-'-^^hiXr,). 

(c) Since tx is monotonically increasing in A, r := lim;^^i tx exists and r < r^. As X is 
left continuous over stopping times, it holds that limx/^iXr)^ = Xr- Because of the 
semicontinuity of i7 and Mv and because v{Xt-^) < jMv^X-,-^) for A /^ 1, we have that 
v{Xt-) = Mv{Xt-). Therefore, r^ < r, i.e. r^ = r. Using dominated concvergence 
and the use of Mv we obtain 

h{x) = liminfE^e"'^'"^/i(X^J < \imini^Exe-'"^^{Mv-J){Xr^) 

= Exe~'^{Mv-J){Xr) < E^e^^^hiXr) = E,e-^"5/i(X,_). 
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Since h is r-superharmonic we obtain 

hix)=E,e-'^shiXrJ. (7) 



■s' 



(d) Note that v{Xr-) = Mv{Xr-). Furthermore, Q yields that condition (pi) from 
Proposition |2 .21 is fulfilled for the impulse control strategy given above. This strategy 



therefore fulfills the requirements of Proposition |2.2| (ii). To see that it is indeed an 



impulse control strategy, note that r^ < Tn+i is indeed fulfilled by Proposition 2.5 
We obtain v{x) = v{x) and the optimality of the impulse control strategy. 

D 

Remark 2.7. Note that the condition ^ is a natural condition to guarantee that the 
value function is finite. For the corresponding equation for optimal stopping problems, 
see the connection in Subsection \3.5\ . 

3. Discussion and examples 

3.1. Impulse control as a convex optimization problem 

It is often convenient to consider optimal stopping problems as linear programming 
problems, see for example |CS02] for a discussion in a general setting. In the same 
line, the previous discussion shows that impulse control problems may be seen as convex 



optimization problems. Indeed, for a fixed state xq G E, we have seen in Theorem 2.3 and 



2.6|that - under some natural conditions - the value v{xq) is given as the the minimum 



of h(xo), where the minimum is taken over all h eH. Note that T-L is indeed convex since 
for all A G [0, 1], /ii, /i2 G 'H for h := Xhi + (1 — A)/i2 > it holds that h is r-superharmonic 
and for all x E E 

{h + J)ix) = Xihi+J)ix) + il-X)ih2 + J)ix) 

>AM(/ii+7)(x) + (l-A)M(/i2+7)(x) 
>M(A(/ii+7) + (l-A)(/i2+7))(x) 
= Mih + J){x), 

so that /i G "H. Therefore, the impulse control problem can be seen as the following 
convex programming problem: 

min h{xo) 

h superharm. 

subj. to {h + J){x) > M{h + J){x) for all xeE. 

3.2. Connection to quasi-variational inequalities 

Now, we can identify the value function as a solution to the corresponding quasi- 
variational inequality under appropriate regularity conditions as follows: As described in 
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the previous section, we can identify h:=v — f as the smallest r-superharmonic function 
with 

M{h + J)<h + J. (8) 

Now, we assume that v is regular enough to apply the generator (or Dynkin operator) 
A oi X. Then we obtain 

> {A-r){v-J){x) = {A-r)v{x)-f{x), 



i.e. {A — r)v — / < 0. Moreover, by (pi) it holds that Mv < v. By the considerations 
leading to ([T]), it is furthermore clear that v — f is r-harmonic on {Mv < v}, i.e. (^ — 
r)f — / = on this set. We obtain that f is a solution to the quasi- variational inequality 

m.a,x{{A — r)v — f,Mv — v} = 0. 

But note that for our approach, no further regularity assumptions on v are needed, see 



also Example 3.4 below 



3.3. One-dimensional diffusion processes 

For the explicit applicability of the theory, it is of interest to have a more explicit 
characterization of the r-superharmonic functions for the process X. Such a character- 
ization is well-known for regular one-dimensional diffusion processes X with absorbing 
or natural boundaries. In this case, a function h is r-superharmonic if and only if ^ is 

^-concave, where 0, ^/^ denote the increasing resp. decreasing fundamental r-harmonic 
functions. We refer to |DK03j for a recent treatment. One could say that the non- 
negative r-superharmonic functions can be characterized as the concave functions in a 
transformed space. Therefore, one can characterize the value function geometrically as 
the smallest nonnegative extended concave function that fulfills M{h + f) <h + f. The 
main difficulty - compared to the optimal stopping problem for one-dimensional diffu- 
sion processes - is that the condition M{h + f) < h-\- f is a nonlocal condition in general, 
since M is a nonlocal operator. Under assumptions that simplify the operator M in a 
suitable way, one can be hopeful to solve the problem geometrically. In some special 
situations, this idea was carried out, see |Ega08| . The main structural results obtained 



there - in our notation - is the following (see EgaOS, Proposition 3.1]): 



Proposition 3.1. Under the assumptions stated in lEga08\l , v — f is the smallest func- 
tion h>0 such that 

{h + J){x) = snpE,e-'^M{h + J){Xr). 

T 

This result can also be seen as the main ingredient used in |Alv04a] . see equation (2.8) 
there. By the general theory of optimal stopping, the previous fact can be stated in the 
following form: Writing 

T-t := {h : E ^ 'Rlh is r-superharmonic, h>0, h-\- f > M{h-\- f), 

\/w r-superharm. with w + f > M{h + f) : w > h}, 
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Proposition 3J^ states that v — f is a pointwise minimizer of H. Obviously, ?/ C H 
On the other hand, it is easily seen - using (JTl) - that v Eli, so that Proposition 3.1 
can be obtained from our general theory in the previous section. In all the examples 
discussed in |Alv04aj . |Ega08| , and |AL08j . the assumptions were chosen such that the 
set S = {x E E : v{x) = Mv{x)} turns out to be essentially one-sided and the process is 
shifted back to one special point. Next, we discuss a (depending on the parameter) one- 
or two-sided problem, that can be dealt with using our approach: 



3.4. Example: Discontinuous costs 

Now, we treat one example in detail, since it deliver insights into the use of r-superharmonic 
functions for the solution of impulse control problems. For simplicity, we consider a stan- 
dard Brownian motion X as an underlying process on ii^ = M. Let / = and assume that 
at each intervention we can shift the process to the state or go on, that is A{x) = {0,x} 
for all a; G M. If we stop at a state x > 1, we receive an amount of 1, and we have to pay 
costs of 1 for a; < 1. In our notation, we have K{x,0) = —1 for a; > 1 and K{x,0) = 1 for 
X < 1 (x 7^ 0). Note that the cost functional is discontinuous at a: = 1, which is not easy 
to handle for the ordinary approaches to impulse control. Nonetheless, the formulation 
using superharmonic function can deal with this problem immediately: 
Now, we discuss how to construct a pointwise minimizer in Ti. First, note that 



Mh{x) = h{0)~K{x,0) 



h{0) + l , x>l, 
h{0)-l , x<l. 



(9) 



For X > 1 it seems to be reasonable to stop immediately and receive the reward. There- 
fore, we make the Ansatz Mh{x) = h{x) for x > 1, i.e. h{x) = h{0) + 1. For x < 1, it is 
not obvious if it is reasonable to shift the process to state 0. Indeed, it depends on the 
discounting parameter r. We distinguish two cases: 





Figure 3.1: Value function for case 1 



Figure 3.2: Value function for case 2 



1. case: e^ > 2, where (3 = ^/2r, i.e. the discounting factor is sufficiently high. 
We make the Ansatz, that h is r-harmonic on (— cxd,1]. Furthermore, it seems to be 
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reasonable, that h is non-increasing on that interval. The general theory of r-harmonic 
functions yields that h(x) = Xe"^ with /3 as above for some A > 0. We see that A = h{0), 
so that we consider 

lA + l, x>l, 



see Figure 3.1 Since r-superharmonic functions for a standard Brownian motion are 



continuous, we should have Ae^ = A-|- 1, i.e. 

A ' 



;/3_l' 



Now, we check that h ^T-i. Obviously, /i > 0. To see that h is r-superharmonic, we 
recognize that h = min{/ii,/i2}, where hi{x) = Ae^^, h2{x) := A+ 1. Since hi and /12 are 
obviously r-superharmonic, so is h. We want to remark, that h is not smooth at x = 1; in 
particular it is not smooth enough to apply Ito's formula (in its standard form without 
local time), as required in most verification theorems for impulse control. It remains to 
be checked that Mh{x) < h{x). Keeping ^ in mind, this is trivial for a; > 1. For x <1, 
we have Mh{x) = A — 1. A short calculation yields that Mh{x) < h{x) iff e^ > 2, which 
is the assumption above. In that case, we indeed have Mh{x) < h{x) for all a; < 1. We 
have proved that h El-i. 



Using Proposition 2.2 (i), we obtain that h>v. Inspired by Theorem 2.6, we define 



Tn = inf{t > Tn-i : h{Xt) = Mh{Xt)} = inf{t > r^_i : Xt > 1}, 
7n = 0. 

This is obviously an admissible impulse control strategy. Since h is r-harmonic on 



(— cxD, 1), it fulfills the requirements of Proposition 2.2 (ii) and we obtain that h>v and 
the impulse control strategy mentioned above is optimal. 

2. case: e^ < 2. In this case it turns out to be optimal to shift back the process to 0, 
whenever X is below a threshold x*, x* < to be found. Analogously to the discussion 
above, we make the Ansatz 

(h{0)-l, x<x*, 

h{x) = I Aie^^ + Aae-^^, x e (x*, 1), 
[/i(0) + l, x>l. 



see Figure 3.2, We find the unknown parameters /i(0), Ai, A2,x* via the conditions 

Ai + A2 = /i(0), Aie'3 + A2e-^ = /i(0) + l 
Aie^^* + A2e-^^* = h{0) - 1, ^^Aie^^* - /3A2e-^^* = 0. 

Indeed, it is not hard to check, that the parameters are uniquely determined by this four 
equations under the assumption e^ < 2. The first two conditions are analogously to the 
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1. case, the second two guarantee that h is smooth at x*, which leads to the conclusion 
that 

//i(0)-l, x<x*, 

h\ (X) ■= { a o 

^' [Aie^^ + Aae-^^, a;G(x*,oo), 

is r-superharmonic. Therefore, so is h with the same argument as in the 1. case. Again, 
we obtain h ETi and using Proposition |2]2 we see that v = h, and 



r„ = inf{t>r„_i:Xt^[x*,l]}, 
7n = 



is optimal. 



3.5. Connection to the ordinary theory of optimal stopping 

The line of argument given above is inspired by the treatment of optimal stopping 
problems, as presented for example in |PS06] . Now, we want to discuss how to find 
the optimal stopping problems as a subclass in the class of optimal impulse control 
problems. In the setting above, assume that E contains a grave d, that is never reached 
by the uncontrolled process when started in E\{d}. If the process reaches d, it stays 
there forever. Then we assume A{x) = {x,d} for x ^ d and Aip) = {d}. Note that 
condition ([s]) is obviously fulfilled. Furthermore, write g{x) := —K{x,d) for all x and 
assume f{d) = 0. For each admissible impulse control S = {Tni'^n) (when ignoring the 
trivial case X-r„ = X^^ _), it holds that 7n = c? for all n. Therefore, 




f{Xs)ds-f2 e-^^"ir(X,„,_,X,J^ =E, (^j\-''f{Xs)ds + e-'^^g{X, 



n=0 

We obtain that we are indeed faced with an ordinary optimal stopping problem with 
discounting and an integral term. Hence, we can consider this class of optimal stopping 
problems as a subclass of the impulse control problems. Letting / = for simplicity and 
noting that h{d) = for all r-superharmonic functions h, we see that the condition 

M{h + J){x)<h + J 

becomes 

g{x) <h{x), 

i.e. h majorizes g. This corresponds to the well-known results for ordinary optimal 
stopping problems. Note that condition ^ in Theorem 2.6 boils down to the standard 
integrability condition 

ExSupe^''^\g{Xt)\ < oo. 

f>0 
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3.6. Multiple stopping problem with random refraction period 

As a generalization of optimal stopping problems, we now consider the following class of 
multiple stopping problems: In the last years, a theory was developed for solving multiple 
stopping problems inspired by applications to swing options in the energy market, see 
|CD08] . |CT08] . and |CIJ12] . In a Markov process setting, these are problems of the 

following form: 

k 

sup E,J2e''^^giY^J, 

where y is a strong Markov process with state space X, g is a, measurable function > 0, 
and the supremum is taken over all stopping times o"i,...,crfc, where it is assumed that 
between each two exercises, there is a refraction period of deterministic length S > 0, 
that is crj_|_i < (Ji + S for all i < k. The main theoretical result was that this problem 
can be reduced to a sequence of n ordinary optimal stopping problems, see |CT08] and 
\CU12\ . 

Now, we will show that this result can also be immediately obtained using the theory 
developed before. To this end, we introduce a new Markov X with state space 

E=i\JXx{i}\u i(jA:x[0,6]x{t}\ U{d} 

as follows: 

• 9 is an absorbing state. 

• Started in a point {x,i) G A" x {i}, i >1, the process does not leave X x {i} and 
has the same dynamics on this space as Y on X. 

• Started in a point {x,s,i) eX x [0,S] x {i}, i>2, the process is given by {x,s + r,i) 
for r & [0,6 — s) and is then restarted in X x {i — 1} with initial distribution Fx{Xs G 



For this process, we specify an impulse control problem as follows: 

A{z) = {z} forall;2G | [J A" x [0,(5] x {i} j U{d}, 

A{x,i) = {{x,i), {x,0,i)} for all {x,i) E X x {i},i > 2, 
A{x,l) = {{x,l),d} for all (x, 1) G A" x {1}, 
K{{x,i),{x,0,i)) = -g{x) = K{{x,l),d) for allxeX,i>2, 

/ = o. 

By the construction of the controlled process, we can identify each impulse control 
strategy {Tn,jn)nG'N for a starting state in A" x {k} with a sequence of stopping times 
cri,...,(Tfc with cTj+i <ai + 6 for all i <k. Therefore, the multiple stopping problem can 
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be identified with the impulse control problem described above. Let v denote the value 
function of the impulse control problem. Then obviously 

v{d) = Q. 

For the maximum operator, it holds that 

Mw{x,l)=w{d)-K{{x,l),d)=w{d)+g{x), x G A", 

Therefore, on A* x {1} v can be found as the smallest r-superharmonic majorant oi g{x), 
this is V is the value function of the optimal ordinary stopping problem for g. Now, write 

h5,i{x) := e-'^¥.^viY5,l){=v{xX^)) , xeX. 

Using this notation, we obtain on A" x {2} that the maximum operator is given by 

Mw{x,2) = w{x,2,0)+g{x) = g{x) + e-^^E:,w{Ys,l). 

We obtain that on A" x {2} the value function v is the smallest r-superharmonic majorant 
of the function 

i.e. V is found to be the value function of the ordinary stopping problem with reward 
function g + hgi. Using induction, we obtain the same result for each k and we see 
that the value function can be found by solving a sequence of ordinary optimal stopping 
problems. 

3.7. Optimal switching problems 

One of the most prominent class of impulse control problems is given by optimal switch- 
ing problems, see e.g. |BL84j . |LB83j . and |BE10j . Using our notations, the problem 
can be stated as follows: 
For two Markov processes X^ \X^^ with joint state space E consider the space 

E = {0,l}xE 

and the stochastic process X with (uncontrolled) distribution as those of {i,X^^') when 
started in {i,x) G {i} x E, i = 0,1. The set of possible controls is then given by 

A{i,x) = {(i,x),(z-l,x)}, 

that is, the decision maker can control the distribution of the underlying process. There- 
fore, the maximum operator is given by 

Mh{i,x) = h{l — i,x) — ki{x), i = 0, 1, x E E, 

where we write ki{x) = K{{i,x),{l — i,x)). Therefore, the transformed value function 
V — f can be characterized as the smallest r-superharmonic function h>0 that fulfills 

{h + J){i,x) > {h + J){l-i,x)-ki{x), i = 0,l, x e E, 

which can be interpreted as a coupled system of two optimal stopping problems. 
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3.8. Application to general Levy processes 

Now, we discuss the case of a general Levy process X on ii^ = M to illustrate that the 
general theory leads to useful results in particularly interesting cases. In this generality, 
there is no hope to find the solution of the optimal impulse control problem explicitly in 
greater generality. Nonetheless, we want to describe the structure of the value function, 
that may be useful as an Ansatz in many concrete situations of interest. The main tool for 
such a representation is the general integral representation of r-superharmonic/excessive 
functions using the Riesz representation theorem. 

We concentrate on the particularly interesting case that an optimal impulse control 
strategy is a constant-boundary strategy, that is there exist a <a < (3 <b such that the 
impulse control strategy S given by 



Tn = inf{t > Tn-1 : Xt ^ {a,b)} 

In 



Xrn,^ < a, 



/, ^r„,- > b. 

There is no hope to find the parameters (a,a,/3,6) and the associated value function 
explicitly in great generality. Even ordinary optimal stopping problems for Levy pro- 
cesses are very hard to solve. A new method developed over the last years is to make 
the Ansatz to write the value function as an expectation of the running maximum or 
minimum of the process evaluated at an independent ii^xp(r)-distributed time T, see e.g. 
jNSOTj . |Mor02j . |Sur07j . |DLU09j . |CST13j . We will show in the following, that one can 
be hopeful to use the same approach also for impulse control problems. 
The main tool is the following representation of general non-negative r-excessive func- 
tions k (under some conditions): 

k{x) = Gr{x,y)a{dy) 

for some Radon measure a = a^, where Gr{x,y) denotes the Green kernel of the process; 
this representation is based on the Riesz representation theorem, see |MS07j and the 
references therein for a more detailed discussion. The measure a does not charge the 
points in M, where k is r-harmonic. Now, we use this representation for k = v — f. By 
equation O we see that v — f is typically r-harmonic on (a, 6). Therefore, a has support 
on {a, by. We may write 



{v-f){x)= Gr{x,y)cx{dy)+ Gr{x,y)a{dy) 

J[—oo,a\ J[b,oo) 

for all X G -E. Writing M = sup{Xt : t < T} and / = mi{Xt :t <T} and assume that M 
and / have densities f'Myfh by the Wiener- Hop f- factorization, we have the representation 

^f X ,S-^fl{t)fM{y-x-t)dt ,y-2;<0, 
rGrix.y) = < ^ 

^I^.jM{t)fi{y-x-t)dt ,y-x>0, 



19 



see |MSn7j . We obtain for all x G {a,b) 



{v-f){x)= Gr{x,y)a{dy)+ Gr{x,y)a{dy) 

j(— cx3,a] J[b,oo) 

=r / / fi{t)fM{y-x-t)dta{dy) 

/•CX3 /-OO 

+ r-^ / fM{t)fi{y-x-t)dta{dy) 

Jb Jy^x 
=r-^ r "^ flit) r fM{y~x-t)a{dy)dt 

J—oo Jx+t 

/•OO fX+t 



+ r ' fhdt) fi{y~x-t)a{dy)dt 

Jb—x Jb 

=E^(g*(/);/ < a) +E,(g*(M);M > b), 



where 



Q,{z) = r-^ r fM{y-z)a{dy), Q\z)=r-^ f fi{y - z)a{dy). 

J z Jb 



This gives a representation of the value function in terms of the running maximum and 
minimum, as desired. On the other hand, functions of the form 



Grix,y)a{dy)+ Gr{x,y)a{dy) 

(—00, a] J[b,oo) 

are r-superharmonic and one can start with these functions to find a candidate solu- 
tion for the value function. This discussion opens the door to use the strong methods 
developed for optimal stopping with underlying Levy processes for impulse control prob- 
lems. Since carrying out the details for a concrete example is quite lengthy, we stop the 
discussion here, but treat an interesting example from portfolio optimization for fixed 
transaction costs for general Levy processes in a forthcoming article. 
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A. On the formal definition of impulse control strategies 

Now, we give a more formal definition of an impulse control strategy and the corre- 
sponding controlled process by following the classical construction, see e.g. |Ste83] to 
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give a reference in English: 

We consider the new probabihty space (l = Vl . Then, 

ri is an LFt® (^{0,^2} stopping time, 

V k>2 J i>o 

71 is J-'n ® (^{0,r2}-mesurable. 
fc>2 

and, more generally, for all n G N 

/n \ 

Tn is an (^ Ji® (^ {0,^} stopping time, 

V/=i k>n+l /i>o 

n 

7„ is (g) J'^„ ® (g) {0,f]} =: J-i;^)-mesurable. 

1=1 k>n+l 

At each random time r^ an impulse is exercised and the process is restarted at the new 
state 7n. Tn and ■jn only depend on the first n coordinates in fl. The restarted processes 
is described by the [n + l)-th coordinate of ^2. More precisely, there exists a family 
Pf , X G E, o( probability measures on Cl, that is characterized by the distributions of 
the coordinate processes as follows: For all s > 0,n G N,Ai,...,An+i measurable, 

pf (x«+. e A,,...,xi:i, G Ar.„xi:xi^ G An+i\H:^) 

=S (i){Ai) ■ ... ■ 6 (n){An)f^^{Xs eAn+i) on {r„ + s<r„+i}, 

where 6 denotes the Dirac measure. The trajectories of the controlled process are then 
given by the trajectories of the copies X"',n = 1,2, ... oi X on Q as follows: 

Xt{uj) =Xf(Wn) for te [Tn-l,Tn), Xr„{u) = 7„(wi, ..., CJn), 10 = 0. 
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